METHOD AND APPARATUS FOR MATCHING POSITIONS OF IMAGES 



BACKGROUND OF THE INVENTION 
Field of the Invention 
5 This invention relates to a method and apparatus for 

matching positions of images. This invention particularly 
relates to an improvement in position matching of two images of 
a single same object. 

Description of the Related Art 

10 Techniques for comparing at least two images of a single 

same object, finding a difference between the* images, and making 
an examination of the object, or the like, in accordance with 
the difference between the images have heretofore been conducted 
in various fields. 

15 For example, in the fields of production of industrial 

products, there have heretofore been conducted techniques for 
comparing an image of a certain product, which image has been 
recorded when the product is in a state of a new article, and 
an image of the product, which image has been recorded after a 

20 durability test has been made on the product, paying attention 

to part of the product, at which the difference between the images 
is large, and determining part of the product, at which the 
durability is to be enhanced. Also, in the medical fields, there 
have heretofore been conducted techniques for comparing a 

25 plurality of radiation images of a certain diseased part of a 

patient, which radiation images have been recorded in a time series 
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mode, with one another, finding the condition of advance of the 
disease or the condition of cure of the disease, and determining 
a plan for treatment of the disease. 

As described above, the techniques for comparing at 
5 least two images are conducted ordinarily in various fields. For 

making the comparison of the at least two images, the at least 
two images are often displayed on an image display device, and 
the like. Specifically, the images having been recorded are 
transformed into image density signals or luminance signals, and 

10 visible images are reproduced from the image density signals or 

luminance signals and displayed on the image display device, and 
the like . Alternatively, visible images are reproduced on a medium, 
such as film, by the utilization of a printer. 

In cases where at least two images, which are to be 

15 compared with each other , are outputted, the images are ordinarily 

outputted so as to stand side by side with each other. In cases 
where the comparison is made, a matter of primary concern for 
the person, who sees the outputted images , is the difference between 
the images . However, it is not easy to find the difference between 

20 the images, which are merely located so as to stand side by side 

with each other. Particularly when the difference is small, it 
is difficult to find the difference in the manner described above. 
Therefore, there is a strong demand for enhancement of performance 
of the comparison. 

25 Accordingly, ordinarily, techniques for performing 

inter-image operation processing, such as a subtraction process. 
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on corresponding pixels in the at least two images to be compared 
with each other, and thereby extracting and enhancing the 
difference between the images are conducted. (One of such 
techniques is proposed in, for example, Japanese Patent 
5 Application No. 11 ( 1999 ) - 342 900 . ) In cases where only the 

difference between the images is thus extracted and enhanced, 
the difference between the images is capable of being reliably 
perceivedby the person, who sees the outputted image . As a result, 
the diseased part, which is advancing or curing, is capable of 

10 being found reliably. 

In cases where the inter- image operation processing 
is to be performed, it is necessary that positions of the two 
images, which are to be subjected to the comparison, i.e. the 
inter - image operation processing, be accurately matched with each 

15 other. If the positions of the two images are not accurately 

matched with each other, an artifact of structures embedded in 
the images will occur from the inter- image operation processing, 
which is performed on the corresponding pixels in the images. 

Besides the images having been recorded successively 

20 in the time series mode, two images, which have been acquired 

at two different points of time apart from each other, are often 
subjected to the comparison. In such cases, ordinarily, a shift 
occurs between the two acquired images due to a deviation in 
orientation of the object during the image recording operation, 

25 error, and the like. Therefore, before the inter-image operation 

processing is performed, it is necessary for the shift to be 
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compensated for. 

Accordingly, approximate position matching processing 
is performed on the entire area of at least either one of the 
two images of the same object, which images are to be compared 
5 with each other . As the approximate posi tion matching processing, 

global position matching processing (a linear transform, e.g. 
an affine transform) , such as rotating processing, parallel 
translation, and image size enlargement or reduction processing, 
may be performed. Alternatively, as the approximate position 

10 matching processing, local position matching processing with 

nonlinear strain transform processing (i.e., warping) in 
accordance with a corresponding position relationship, which has 
been obtained from template matching for each of local area limited 
regions, may be performed. The nonlinear strain transform 

15 processing may be performed with, for example, curve fitting 

utilizing two-dimensional polynomials. As another alternative, 
as the approximate position matching processing, a combination 
of the global position matching processing and the local position 
matching processing may be performed . (Techniques for performing 

20 the approximate position matching processing are described in, 

for example, Japanese Unexamined Patent Publication Nos . 
7(1995) - 37074 and 8(1996) - 335721.) With the approximate posi tion 
matching processing described above, position matching is capable 
of being effected to a certain extent with respect to the entire 

25 areas of the images. 

However, for example, in cases where image patterns 
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of similar textures are embedded in adjacent local area limited 
regions as in the cases of rib image patterns in chest radiation 
images, the approximate position matching processing described 
above has the problems in that a shift often remains between certain 
5 local area limited regions. 

SUMMARY OF THE INVENTION 
The primary ob j ect of the present invention is to provide 
a method of matching positions of images, wherein a shift between 
two images to be compared wi th each other i s suppressed more r el i ably 
10 than with conventional image position matching techniques, arid 

position matching is performed accurately. 

Another object of the present invention is to provide 
an apparatus for carrying out the method of matching positions 
of images. 

15 A method and apparatus for matching positions of images 

in accordance with the present invention are characterized by 
performing approximate position matching processing with respect 
to entire areas of two images, selecting certain local area limited 
regions, between which a degree of shift is high even after the 

20 approximate position matching processing has been performed, and 

performing position re-matching processing with respect to only 
the selected local area limited regions and, when necessary, 
regions neighboring with the selected local area limited regions, 
such that the degree of shift between the local area limited regions , 

25 between which the degree of shift was high, may be suppressed, 

and such that the degree of shift between the entire areas of 
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the images may be suppressed more reliably than with conventional 
image position matching techniques. 

Specifically, the present invention provides a method 
of matching positions of images, in which positions of two images 
of a single same object are matched with each other, the method 
comprising the steps of: 

i) performing approximate position matching 
processing with respect to entire areas of the two images, 

ii) selecting local area limited regions, between 
which a degree of shift is high, in the two images, whose positions 
have been approximately matched with each other by the approximate 
position matching processing, and 

iii) performing position re-matching processing with 
respect to at least the local area limited regions, which have 
thus been selected. 

In the method of matching positions of images in 
accordance with the present invention, the object may be one of 
various obj ects , such as human bodies , animals, plants, industrial 
products, geographical features, celestial structures, and 
scenes . 

Also, the two images of the single same object should 
preferably be the images subjected to inter- image operation 
processing, in which the position matching should be performed 
at a high accuracy. The inter-image operation processing should 
preferably be a subtraction process performed on corresponding 
pixels in the two images. In such cases, the subtraction process 



may be a simple subtraction or a weighted subtraction. An 
inter-image operation processed image, which has been obtained 
from the subtraction process, is ordinarily referred to as the 
subtraction image. By way of example, the subtraction image may 
5 be an energy subtraction image obtained with the simple subtraction 

or the weighted subtraction performed on two original images, 
which have been formed with radiation having different energy 
distributions and approximately simultaneously (one of which is 
a high energy image having been formed with radiation having a 

10 high energy level , i.e. an ordinary radiation image, and the other 

of which is a low energy image having been formed with radiation 
having a low energy level, i.e. a high energy suppressed image) . 
Alternatively, the subtraction image may be a temporal (time 
difference) subtraction image obtained from two original images, 

15 which have been recorded at two different points of time apart 

from each other. As another alternative, the subtraction image 
may be a digital subtraction angiography image obtained from two 
original images of blood vessels, which images have been recorded 
before injection of contrast media and after injection of the 

20 contrast media. 

As described above, the two images of the single same 
object may be, for example, the two radiation images of an identical 
site of the single same object, which images have been recorded 
at two different points of time apart from each other. However, 

25 the two images of the single same object are not limited to the 

two radiation images described above, and the like. For example, 
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the two images of the single same object may be two image areas, 
which are embedded in a single image and which represent sites 
having approximately identical shapes (e.g., the right and left 
lungs, or the right and left mammae) . Specifically, the two images 
5 are not limited to two different images and may be two different 

image areas embedded in a single same image. 

The two images described above should preferably be 
the medical radiation images. 

In the method of matching positions of images in 
10 accordance with the present invention, the approximate position 

matching processing should preferably be global position matching 
processing for performing transform processing (such as an af f ine 
transform) comprising at least one kind of processing, which is 
among rotating processing, parallel translation, and image size 
15 enlargement or reduction processing, on an entire area of at least 

either one of the two images. 

Alternatively, in the method of matching positions of 
images in accordance with the present invention, the approximate 
position matching processing should preferably be local position 
20 matching processing for: 

setting a plurality of template regions , which are small 
regions, in one of the two images, 

setting a plurality of search regions, each of which 
corresponds to one of the template regions having been set in 
25 the one image and is larger than each of the template regions, 

in the other image. 
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determining a subregion in each of the search regions, 
in which subregion an image pattern approximately coincides with 
the image pattern within the corresponding template region, and 
performing nonlinear strain transform processing (e.g. , 
5 a nonlinear strain transform utilizing curve fitting with 

two-dimensional polynomials) on the entire area of at least either 
one of the two images and in accordance with corresponding position 
relationships between the subregions, which have thus been 
determined, and the corresponding template regions, such that 

10 the two images approximately coincide with each other. 

With the global position matching processing or the 
local position matching processing described above, as the 
approximate position matching with respect to the entire areas 
of the two images, shifts between the two images are capable of 

15 being suppressed to a certain extent. As another alternative, 

in the method of matching positions of images in accordance with 
the present invention, the approximate position matching 
processing may be a combination of the global position matching 
processing described above and the local position matching 

20 processing described above, which is performed after the global 
position matching processing. (Examples of the techniques for 
performing the combination of the global position matching 
processing and the local position matching processing are 
described in, for example, Japanese Unexamined Patent Publication 

25 Nos.7(1995)-37074 and 8(1996) - 335721.) As the subregion in each 

of the search regions # for example , a subregion , which is associated 
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with the largest cross correlation value between the subregion 
and the corresponding template region, may be determined. 
Alternatively, as the subregion in each of the search regions, 
a subregion, which is associated with the smallest value of the 
5 total sum of absolute values of pixel value differences between 

the subregion and the corresponding template region, may be 
determined . 

Also, in the method of matching positions of images 
in accordance with the present invention, the selection of the 

10 local area limited regions, between which the degree of shift 

is high, may be performed by manually and arbitrarily setting 
the local area limited regions, between which the degree of shift 
is high, in the two images. Alternatively, the selection of the 
local area limited regions, between which the degree of shift 

15 is high, may be performed by automatically setting a plurality 

of local area limited regions over the entire area of each of 
the two images regardless of the degree of shift, calculating 
the degree of shift between corresponding local area limited 
regions in the two images with respect to each of sets of the 

20 corresponding local area limited regions in the two images, and 

automatically selecting the local area limited regions, between 
which the degree of shift is high, in the two images in accordance 
with the thus calculated degrees of shift. Such that the labor 
for the manual setting may be eliminated, the selection of the 

25 local area limited regions, between which the degree of shift 

is high, should preferably be performed automatically. In such 
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cases, the degree of shift between the corresponding local area 
limited regions should preferably be calculated as a total sum 
value of absolute values of pixel value differences between 
corresponding pixels in the corresponding local area limited 
5 regions in the two images, and the selection of the local area 

limited regions, between which the degree of shift is high, should 
preferably be made by performing threshold value processing on 
the total sum value of the absolute values of the pixel value 
differences between the corresponding pixels in the corresponding 

10 local area limited regions . Alternatively, as the degree of shift , 

one of other index values (e.g., a normalized cross correlation 
value) may be employed. By the utilization of the index value, 
the local area limited regions, between which the degree of shift 
is high, may be selected. Further, as the corresponding local 

15 area limited regions in the two images, regions having an identical 

center point and an identical shape (e.g., rectangular regions) 
in the two images may be employed. In cases where the local area 
limited regions, between which the degree of shift is high, are 
to be selected manually , a technique should preferably be employed 

20 wherein, when a certain local area limited region is set in one 

of the two images, a local area limited region in the other image, 
which region has spatial coordinates identical with the spatial 
coordinates of the certain local area limited region having been 
set in the one image and has a size identical with the size of 

25 the certain local area limited region having been set in the one 

image, is set automatically . Ordinarily, i t i s not easy to manual ly 
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and accurately set the regions having the identical spatial 
coordinates and the identical size in the two images. 

In the method of matching positions of images in 
accordance with the present invention, the position re-matching 
5 processing is performed with respect to at least the selected 

local area limited regions. Specifically, the position 
re-matching processing is performed primarily with respect to 
the selected local area limited regions. However, in cases where 
only the selected local area limited regions are subjected to 

10 the position re - matching processing, if local area limited regions 

adjacent to the selected local area limited regions are not selected 
for the position re-matching processing, there will be the risk 
that a shift will occur at the boundary between the local area 
limited region in one image, which region has been selected and 

15 sub j ec ted to the position re - matching processing , and the ad j acent 

local area limited region in the image, which region has not been 
selected and has not been subjected to the position re-matching 
processing. Therefore, when necessary, besides the selected 
local area limited regions, regions neighboring with the selected 

20 local area limited regions may also be subjected to the position 

re-matching processing. The term "performing position 

re-matching processing with respect to at least selected local 
area limited regions" as used herein does not mean that the images 
containing the local area limited regions , which have been selected, 

25 and all of the local area limited regions, which have not been 

selected, are subjected as a whole to the position re-matching 
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processing . 

In the method of matching positions of images in 
accordance with the present invention, the position re-matching 
processing should preferably be a processing for: 
5 setting a template region in the local area limited 

region in one of the two images, the setting of the template region 
being performed with respect to each of sets of the corresponding 
local area limited regions in the two images, between which local 
area limited regions the degree of shift is high, 

10 setting a search region in the local area limited region 

in the other image, the setting of the search region being performed 
with respect to each of the sets of the corresponding local area 
limited regions in the two images , between which local area limited 
regions the degree of shift is high, 

15 determining a subregion within the search region, in 

which subregion a degree of image pattern coincidence with the 
corresponding template region is high, 

calculating a corresponding position relationship 
between the template region and the corresponding subregion, 

20 iterating operations, in which sizes of the template 

region and the search region are reduced in stages, the template 
region and the search region having the sizes reduced in stages 
are set in accordance with the thus calculated corresponding 
position relationship, and a corresponding position relationship 

25 between the template region and the corresponding subregion is 

calculated successively, an ultimate corresponding position 
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relationship being thereby calculated with respect to each of 
the sets of the corresponding local area limited regions in the 
two images, between which local area limited regions the degree 
of shift is high, and 
5 performing transform processing on at least either one 

of the two images , whose positions have been approximately matched 
with each other by the approximate position matching processing, 
and in accordance with the ultimate corresponding position 
relationships that have been calculated ultimately with respect 

10 to the sets of the corresponding local area limited regions in 

the two images , between which local area limited regions the degree 
of shift is high, such that the corresponding local area limited 
regions in the two images , between which local area 1 imi ted regions 
the degree of shift is high, approximately coincide with each 

15 other. 

In the manner described above, the sizes of the template 
region and the search region may be reduced in stages, the 
corresponding position relationship may be calculated 
successively such that the degree of image pattern coincidence 

20 may become high, and the position matching may be performed in 

accordance with the calculated corresponding position 
relationship. In such cases, the image as a whole is capable of 
being prevented from distorting unnaturally, and the shift between 
the two images is capable of being suppressed more reliably than 

25 with the conventional image position matching techniques. Also, 

the shift between the two images is capable of being eliminated 
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accurately . 

Specifically, the accuracy of the position matching 
between the local area limited regions is capable of being enhanced 
by calculating the corresponding positions in a good position 
5 matching state (i.e., in a high degree of coincidence) by the 

utilization of the small regions (i.e., the template region and 
the search region) . In such cases, in order for the position 
matching accuracy to be enhanced, the sizes of the utilized regions 
should preferably be as small as possible. However, if the small 

10 regions are merely utilized for calculating the corresponding 

positions in a good position matching state (i.e., the optimum 
corresponding positions) , a local minimum phenomenon, wherein 
the highest degree of coincidence is obtained with respect to 
a subregion, which is not the correct optimum corresponding 

15 position, within the search region, will often occur. In such 

cases, the problems will occur in that the optimum corresponding 
position, which is calculated for each of the local area limited 
regions, becomes a position which is capable of being obtained 
only when the local area limited region is translated in a direction 

20 markedly different from the direction of translation of an adj acent 

local area limited region. As a result, the image pattern within 
the local area limited region will distort unnaturally . Therefore, 
in the method of matching positions of images in accordance with 
the present invention, as described above , in order for the optimum 

25 corresponding positions to be calculated, the corresponding 

position relationships should preferably be calculated in stages 
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by reducing the sizes of the regions in stages. In this manner, 
the local minimum phenomenon is capable of being prevented from 
occurring, and the image pattern within the local area limited 
region is capable of being prevented from distorting unnaturally. 
5 The present invention also provides an apparatus for 

carrying out the method of matching positions of images in 
accordance with the present invention . Specifically, the present 
invention also provides an apparatus for matching positions of 
images, in which positions of two images of a single same object 
10 are matched with each other, the apparatus comprising: 

i) approximate position matching processing means for 
performing approximate position matching processing with respect 
to entire areas of the two images, 

ii) region selecting means for selecting local area 
15 limited regions, between which a degree of shift is high, in the 

two images, whose positions have been approximately matched with 
each other by the approximate position matching processing, and 

iii) position re-matching processing means for 
performing position re-matching processing with respect to at 

20 least the local area limited regions, which have been selected 

by the region selecting means. 

In the apparatus for matching positions of images in 

accordance with the present invention, the approximate position 

matching processing means should preferably be global position 
25 matching processing means , which performs global positionmatching 

processing for performing transform processing comprising at least 
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one kind of processing, which is among rotating processing, 
parallel translation, and image size enlargement or reduction 
processing, on an entire area of at least either one of the two 
images . 

5 Alternatively, in the apparatus for matching positions 

of images in accordance with the present invention , the approximate 
position matching processing means should preferably comprise: 

a) template matching processing means for: 

setting a plurality of template regions, which 

10 are small regions, in one of the two images, 

setting a plurality of search regions, each of 
which corresponds to one of the template regions having been set 
in the one image and is larger than each of the template regions, 
in the other image, and 

15 determining a subregion in each of the search 

regions, in which subregion an image pattern approximately 
coincides with the image pattern within the corresponding template 
region, and 

b) local position matching processing means, which 
20 performs local position matching processing for performing 

nonlinear strain transform processing on the entire area of at 
least either one of the two images and in accordance with 
corresponding position relationships between the subregions and 
the corresponding template regions, the corresponding position 
25 relationships having been obtained from the template matching 

processing means, such that the two images approximately coincide 
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with each other. 

As another alternative, in the apparatus for matching 
positions of images in accordance with the present invention, 
the approximate position matching processing means may be a 
5 combination of the global position matching processing means 

described above, the template matching processing means described 
above, and the local position matching processing means described 
above . 

Also, in the apparatus for matching positions of images 

10 in accordance with the present invention, the region selecting 

means may be constituted such that the selection of the local 
area limited regions, between which the degree of shift is high, 
is performed by manually and arbitrarily setting the local area 
limited regions, between which the degree of shift is high, in 

15 the two images. Alternatively, the region selecting means may 

be constituted such that the selection of the local area limited 
regions, between which the degree of shift is high, is performed 
by automatically setting a plurality of local area limited regions 
over the entire area of each of the two images regardless of the 

20 degree of shift, calculating the degree of shift between 

corresponding local area limited regions in the two images with 
respect to each of sets of the corresponding local area limited 
regions in the two images, and automatically selecting the local 
area limited regions, between which the degree of shift is high, 

25 in the two images in accordance with the thus calculated degrees 

of shift. Such that the labor for the manual setting may be 
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eliminated, the region selecting means should preferably be 
constituted such that the selection of the local area limited 
regions, between which the degree of shift is high, is performed 
automatically. In such cases, the region selecting means should 
5 more preferably be constituted such that the degree of shift between 

the corresponding local area limited regions is calculated as 
a total sum value of absolute values of pixel value differences 
between corresponding pixels in the corresponding local area 
limited regions in the two images, and the selection of the local 
10 area limited regions, between which the degree of shift is high, 

is made by performing threshold value processing on the total 
sum value of the absolute values of the pixel value differences 
between the corresponding pixels in the corresponding local area 
limited regions. 

15 In the apparatus for matching positions of images in 

accordance with the present invention, the position re-matching 
processing means should preferably comprise: 

a) staged template matching processing means for: 

setting a template region in the local area limited 
20 region in one of the two images, the setting of the template region 

being performed with respect to each of sets of the corresponding 
local area limited regions in the two images, between whiqh local 
area limited regions the degree of shift is high, 

setting a search region in the local area limited 
25 region in the other image, the setting of the search region being 

performed with respect to each of the sets of the corresponding 
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local area limited regions in the two images, between which local 
area limited regions the degree of shift is high, 

determining a subregion within the search region, 
in which subregion a degree of image pattern coincidence with 
5 the corresponding template region is high, 

calculating a corresponding position 

relationship between the template region and the corresponding 
subregion, and 

iterating operations, in which sizes of the 

10 template region and the search region are reduced in stages, the 

template region and the search region having the sizes reduced 
in stages are set in accordance with the thus calculated 
corresponding position relationship, and a corresponding position 
relationship between the template region and the corresponding 

15 subregion is calculated successively, in order to calculate an 

ultimate corresponding position relationship with respect to each 
of the sets of the corresponding local area limited regions in 
the two images, between which local area limited regions the degree 
of shift is high, and 

20 b) partial position matching processing means for 

performing transform processing on at least either one of the 
two images, whose positions have been approximately matched with 
each other by the approximate position matching processing, and 
in accordance with the ultimate corresponding position 

25 relationships that have been calculated ultimately with respect 

to the sets of the corresponding local area limited regions in 
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the two images, between which local area limited regions the degree 
of shift is high, such that the corresponding local area limited 
regions in the two images , between which local area limited regions 
the degree of shift is high, approximately coincide with each 
5 other. 

In such cases, the staged template matching processing 
means may be means for: 

setting a plurality of small regions around each of 
the corresponding local area limited regions in the two images, 

10 between which local area limited regions the degree of shift is 

high, in at least one stage among the stages associated with the 
operation for reducing the sizes of the template region and the 
search region in stages, 

calculating a corresponding position relationship 

15 between the corresponding small regions in the two images, the 

calculation being made with respect to each of sets of the 
corresponding small regions in the two images, 

constraining the template region and the small regions , 
which surround the template region, with one another, and/or 

20 constraining the corresponding subregion and the small regions, 

which surround the subregion, with one another, the constraint 
being performed with constraining forces in accordance with the 
corresponding position relationship, which has been calculated 
wi th respect to the template region and the corresponding subregion, 

25 and the corresponding position relationships, which have been 

calculated with respect to the sets of the corresponding small 



21 



regions in the two images, 

calculating a new relative corresponding position 
relationship with respect to the template region and the 
corresponding subregion in accordance with the constraining forces , 
5 and 

utilizing the thus calculated new relative 
corresponding position relationship as the corresponding position 
relationship between the template region and the corresponding 
subregion in each of the stages. 

10 The term "plurality of small regions" as used herein 

means various regions, which are capable of being located around 
each local area limited region. For example, in cases where the 
local area limited region has a rectangular shape, the small regions 
may be eight rectangular small regions, which have sizes 

15 approximately identical with the size of the local area limited 

region and surround the local area limited region in a onefold 
manner. Alternatively, in such cases, the small regions may be 
16 rectangular small regions, which have sizes approximately 
identical with the size of the local area limited region and surround 

20 the local area limited region in a twofold manner. 

As described above, the constraint is performed with 
the constraining forces in accordance with the corresponding 
position relationship, which has been calculated with respect 
to the template region and the corresponding subregion, and the 

25 corresponding position relationships , which have been calculated 

with respect to the sets of the corresponding small regions in 
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the two images . (Each of the template region and the corresponding 
subregion will hereinbelow be referred to as the center region.) 
Specifically, at least either one of the center regions is 
constrained with the constraining forces, which are defined by 
5 differences between the corresponding position relationship with 

respect to the center regions and the corresponding position 
relationships with respect to the sets of the corresponding small 
regions in the two images, or which are defined by results of 
other operation processing. For example, as the constraining 

10 force, it is possible to employ an elasticity force, such as a 

spring force, which is obtained in cases where the difference 
between the corresponding position relationship with respect to 
the center regions and the corresponding position relationship 
with respect to each of the sets of the corresponding small regions 

15 in the two images is taken as "displacement quantity." 

Alternatively, as the constraining force, it is possible to employ 
an attractive force, such as a coulomb force, which is obtained 
in cases where the corresponding position relationship with 
reS pect to the center regions and the corresponding position 

20 relationship with respect to each of the sets of the corresponding 

small regions in the two images are taken as "electric charges" 
or "magnetic charges . " In particular, such that the calculations 
may be conducted quickly, the elasticity force undergoing linear 
proportion should preferably be employed as the constraining 

25 force. 

The elasticity modulus should preferably be variable 
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with respect to each of the surrounding small regions . For example, 
the elasticity modulus may be altered in accordance with the 
distance between the center region and each of the surrounding 
small regions. 

5 Also, as described above, a calculation is made in 

accordance with the constraining forces to find the new relative 
corresponding position relationship with respect to the template 
region and the corresponding subregion. Specifically, for 
example, in cases where the center region and the surrounding 

10 small regions are constrained with one another by the constraining 

forces with the springs, a value of the corresponding position 
relationship, by which the center region is pulled back due to 
the constraining forces (elasticity forces) with the springs, 
may be subtracted from the original value of the corresponding 

15 position relationship. In this manner, the new relative 

corresponding position relationship with respect to the template 
region and the corresponding subregion may be calculated. The 
new relative corresponding position relationship with respect 
to the template region and the corresponding subregion may be 

20 calculated in the same manner also in cases where the attractive 

force, such as the coulomb force, is employed as the constraining 
force in lieu of the elasticity force. 

With the method and apparatus for matching positions 
of images in accordance with the present invention, the approximate 

25 position matching processing is performed with respect to the 

entire areas of the two images, and only the certain local area 
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limited regions, between which the degree of shift is high even 
after the approximate position matching processing has been 
performed, are selected. Also, the position re-matching 
processing is performed with respect to only the selected local 
5 area limited regions and, when necessary, the regions neighboring 

with the selected local area limited regions . Therefore, in cases 
where image patterns of similar textures are embedded in adjacent 
local area limited regions as in the cases of rib image patterns 
in chest radiation images, and a shift remains between certain 

10 local area limited regions after the approximate position matching 

processing has been performed, the degree of shift between the 
local area limited regions, between which the degree of shift 
was high, is capable of being suppressed . Accordingly, the degree 
of shift between the entire areas of the images is capable of 

15 being suppressed more reliably than with the conventional image 

position matching techniques, and the position matching accuracy 
is capable of being enhanced. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1 is a flow chart showing a processing flow in 

20 an embodiment of the method of matching positions of images in 

accordance with the present invention, 

Figure 2 is a block diagram showing a first embodiment 
of the apparatus for matching positions of images in accordance 
with the present invention, which carries out the processing flow 

25 shown in Figure 1, 

Figures 3A and 3B are schematic views showing two images 
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PI and P2, whose positions are to be matched with each other by 
the embodiment of Figure 2, 

Figures 4A and 4B are explanatory views showing how 
global position matching processing is performed, 
5 Figures 5A and 5B are explanatory views showing how 

local position matching processing is performed, 

Figure 6 is an explanatory view showing how nonlinear 
strain transform processing is performed, 

Figure 7 is a flow chart showing processing for selecting 
10 regions, which are to be subjected to position re-matching 

processing , 

Figure 8A and 8B are explanatory views showing local 
area limited regions , which are set for the selection of the regions 
to be subjected to the position re-matching processing, 
15 Figure 9 is a block diagram showing position re-matching 

processing means. 

Figure 10 is a flow chart showing the position 
re -matching processing , 

Figures 11A and 11B are explanatory views showing 
20 correspondence relationships among a first template region TR2 , 

a first search region RR2 , and a first subregion tR2 , 

Figures 12A and 12B are explanatory views showing 
correspondence relationships among a second template region TR 1 2 , 
a second search region RR 1 2 , and a second subregion tR'2, 
25 Figure 13 is an explanatory view showing template 

regions TR in a second image P2 , 



26 



Figure 14 is an explanatory view showing a linear 
interpolating operation, in which four neighboring points are 
utilized, 

Figure 15 is an explanatory view showing linear 
interpolating processing in a two-dimensional plane, in which 
four neighboring points are utilized, and which is performed with 
respect to a pixel in a first image Pl° having been obtained from 
the position re-matching processing, 

Figure 16A is a schematic view showing the first image 



10 PI 



P2, 



Figure 16B is a schematic view showing the second image 



Figure 16C is a schematic view showing a subtraction 
image Psu having been obtained from the first image Pl° of Figure 
15 16A and the subtraction image Psu of Figure 16B, 

Figure 17 is a flow chart showing position re-matching 
processing, which is performed in a second embodiment of the 
apparatus for matching positions of images in accordance with 
the present invention, and 
20 Figure 18 is a conceptual view showing how a shift vector 

of a template ROI at a center region and shift vectors of surrounding 
template ROI ' s are constrained with one another by virtual springs . 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The present invention will hereinbelow be described 
25 in further detail with reference to the accompanying drawings. 

Figure 1 is a flow chart showing a processing flow in 
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an embodiment of the method of matching positions of images in 
accordance with the present invention . Figure 2 is a block diagram 
showing a first embodiment of the apparatus for matching positions 
of images in accordance with the present invention, which carries 
5 out the processing flow shown in Figure 1. Figures 3A and 3B are 

schematic views showing two images PI and P2, whose positions 
are to be matched with each other by the embodiment of Figure 
2 . A first image PI and a second image P2 are chest images (radiation 
images) of a single same patient, which images have been recorded 

10 at different points of time. The first image PI is a past image, 

which has been recorded at a point of time earlier than the recording 
of the second image P2 . The second image P2 is a current image, 
which is newer than the first image PI. The first image PI and 
the second image P2 are the images to be subjected to temporal 

15 subtraction processing , wherein a subtraction process is performed 

on corresponding pixels in the two images, and a temporal 
subtraction image is thereby acquired. 

In cases where the two images PI and P2 are compared 
with each other, ordinarily, the person who sees the images, such 

20 as a medical doctor, makes a comparison by taking the second image 

P2, which is the current image, as a reference image. Therefore, 
in this embodiment of the apparatus for matching positions of 
images in accordance with the present invention, the second image 
P2 is taken as the reference image. However, alternatively, the 

25 first image PI may be taken as the reference image, and the first 

image PI and the second image P2 may be replaced by each other 
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in the explanations described below. 

With reference to Figure 2, an image position matching 
apparatus 10 comprises approximate position matching processing 
means 14 for performing approximate position matching processing 
5 with respect to entire areas of the two images PI and P2 of the 

single same object. The image position matching apparatus 10 also 
comprises region selecting means 15 for selecting local area 
limited regions, between which the degree of shift is high, in 
an image PI" (an image having been obtained by processing the 

10 first image PI for the approximate position matching processing) 

and the second image P2, whose positions have been approximately 
matched with each other by the approximate position matching 
processing. The image position matching apparatus 10 further 
comprises position re-matching processing means 16 for performing 

15 position re-matching processing with respect to the local area 

limited regions, which have been selected by the region selecting 
means 15, and regions neighboring with the local area limited 
regions . 

The approximate position matching processing means 14 
20 comprises global position matching processing means 11, which 

performs global position matching processing on the two images 
PI and P2 by performing an affine transform, such as rotating 
processing, parallel translation, and image size enlargement or 
reduction processing, on the entire area of the first image PI, 
25 which has been acquired at a point of time earlier than the second 

image P2 . The approximate position matching processing means 14 
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also comprises template matching processing means 12. The 
template matching processing means 12 sets a plurality of template 
regions T, T, . .., which are rectangular small regions, in the 
second image P2 , which is the current image acting as the reference 
5 image. Specifically, of animagePl 1 ( an image having been obtained 

by performing the processing, i.e., the affine transform, on the 
first image PI for the global position matching processing) and 
the second image P2 , whose positions have been matched with each 
other by the global position matching processing, the second image 

10 P2 is taken as the reference image. The template matching 

processing means 12 also sets a plurality of search regions R, 
R, . . . , each of which corresponds to one of the template regions 
T, T, ... and is larger than each of the template regions T, T, 
in the first image PI 1 having been obtained from the affine transform 

15 The template matching processing means 12 further determines 

subregions T ' , T 1 , ... in the respective search regions R, R, . . . , 
in each of which subregions the image pattern approximately 
coincides with the image pattern within the corresponding template 
region T. Furthermore, the template matching processing means 

20 12 calculates corresponding position relationships between the 

subregions T 1 , T 1 , ... and the corresponding template regions 
T , T , ... The approximate position matching processing means 14 
further comprises local position matching processing means 13, 
which performs local position matching processing in local area 

25 limited units with respect to the entire areas of the first image 

PI 1 and the second image P2 by performing a nonlinear transform 
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(e.g., nonlinear strain transform, i.e. warping, which is curve 
fitting utilizing two-dimensional polynomials of the tenth order) 
on the entire area of the first image PI 1 and in accordance with 
the corresponding position relationships between the subregions 
5 T', ... and the corresponding template regions T, T, 

the corresponding position relationships having been obtained 
from the template matching processing means 12, such that the 
subregions T 1 , T 1 , ... in the first image PI 1 approximately coincide 
with the corresponding template regions T, T, ... in the second 
10 image P2 . 

As illustrated in Figures 8 A and 8B , the region selecting 
means 15 sets a plurality of regions of interest (ROI's) on each 
of the first image PI" and the second image P2 , whose positions 
have been approximately matched with each other by the approximate 

15 position matching processing . The ROI 1 s have their center points 

at corresponding spatial coordinates on the first image P1 M and 
the second image P2 and have an identical shape, e . g . a rectangular 
shape having a size of 16 pixels (in the vertical direction) x 
16 pixels (in the horizontal direction) . Specifically, ROI1, ROI2, 

20 ROI3 , . . . are set in the second image P2 , andROIl 1 , ROI2 ' , ROI3 ' , . . . 

are set in the first image PI". Also, with respect to each of 
sets of the corresponding ROI's (e.g., a set of ROI1 and R0I1', 
a set of ROI2 and ROI2 1 , and so on) in the second image P2 and 
the first image PI", the region selecting means 15 calculates 

25 a total sum value Ds of absolute values of pixel value differences 

between corresponding pixels (i.e., the pixels having identical 
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spatial coordinates) in the corresponding ROI 1 s as an index 
representing the degree of shift between the corresponding ROI 1 s . 
Specifically, as the index representing the degree of shift between 
the corresponding ROI 1 s, the region selecting means 15 calculates 
5 the total sum value Ds [«EL|A (i , j ) -B (i, j ) |] of the absolute values 

of the differences [=|A(i, j)-B(i, j ) |] between the pixel values 
of the corresponding pixels [a pixel value A(i, j) of the pixel 
within the ROI in the second image P2 and a pixel value B(i, j) 
of the corresponding pixel within the corresponding ROI in the 

10 first image PI" after being transformed] . Also, the region 

selecting means 15 compares the thus obtained total sum value 
Ds with a threshold value K having been set previously. The region 
selecting means 15 selects the corresponding ROI's, which are 
associated with the total sum value Ds larger than the threshold 

15 value K (Ds>K) , as the local area limited regions, between which 

the degree of shift is high, and which are to be subjected to 
the position re-matching processing. Also, the region selecting 
means 15 regards the corresponding ROI's, which are associated 
with the total sum value Ds not larger than the threshold value 

20 K (Ds^K) , as the local area limited regions, between which the 

degree of shift is low, and which are not to be subjected to the 
position re-matching processing. 

As illustrated in Figure 9, the position re-matching 
processing means 16 comprises staged template matching processing 

25 means 16a and partial position matching processing means 16b. 

The staged template matching processing means 16a sets a template 
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region (i.e., a template ROI) in one of the corresponding ROI's, 
between which the degree of shift is high, and which have been 
selected by the region selecting means 15. The staged template 
matching processing means 16a also sets a search region (i.e., 
5 a search ROI) in the other ROI . The setting of the template region 

and the search region is performed with respect to each of the 
sets of the corresponding ROI 1 s , between which the degree of shift 
is high. Further, the staged template matching processing means 
16a determines a subregion within the search region, in which 

10 subregion the degree of image pattern coincidence with the 

corresponding template ROI is high. Furthermore, the staged 
template matching processing means 16a calculates a corresponding 
position relationship (i.e., a shift quantity) between the 
template region and the corresponding subregion. Thereafter, the 

15 staged template matching processing means 16a iterates operations , 

in which the sizes of the template region and the search region 
are reduced in stages, the template region and the search region 
having the sizes reduced in stages are set in accordance with 
the corresponding position relationship having been calculated 

20 in the preceding stage, and a corresponding position relationship 

between the template region and the corresponding subregion is 
calculated successively. In this manner, the staged template 
matching processing means 16a calculates an ultimate corresponding 
position relationship with respect to each of the sets of the 

25 corresponding ROI's, between which the degree of shift is high. 

(In this embodiment, an accumulated corresponding position 
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relationship, which is obtained by accumulating the corresponding 
position relationships, i.e. the shift quantities, at the 
respective stages, is employed as the ultimate corresponding 
position relationship.) In accordance with the ultimate 
5 corresponding position relationships, which have been calculated 

ultimately by the staged template matching processing means 16a 
and with respect to the sets of the corresponding ROI's, between 
which the degree of shift is high, the partial position matching 
processing means 16b performs transform processing for the 

10 position matching on the local area limited region in the first 

image PI", which local area limited region has been selected, 
and regions neighboring with the local area limited region, such 
that the corresponding local area limited regions, between which 
the degree of shift is high, in the first image PI" and the second 

15 image P2 , whose positions have been approximately matched with 

each other by the approximate position matching processing, 
approximately coincide with each other. 

Specifically, as illustrated in Figure 10, the staged 
template matching processing means 16a performs the processing 

20 described below. 

(1) As for the second image P2 acting as the reference 
image, with respect to each ROI in the second image P2, the staged 
template matching processing means 16a sets a first template region 
TR (one of TR1 , TR2 , . . . ) , which has its center point at a center 

25 point having coordinates (xO, yO) in the ROI. By way of example, 

the first template region TR may have a rectangular shape having 
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a size of 80 pixels (in the vertical direction) * 80 pixels (in 
the horizontal direction) . Also, as for the first image PI", with 
respect to each ROI in the first image P1 M , the staged template 
matching processing means 16a sets a first search region RR (one 
5 of RR1 , RR2 , . . . ) , which has its center point at the same coordinates 

(xO, yO) as the coordinates of the center point of the firs t template 
region TR and is larger than the first template region TR. By 
way of example, the first search region RR may have a rectangular 
shape having a size of 92 pixels (in the vertical direction) * 

10 92 pixels (in the horizontal direction) . Further, with respect 

to each of the sets of the first template regions TR, TR, . . . 
and the corresponding first search regions RR, RR, ... (a set 
of TR1 and RR1 , a set of TR2 and RR2 , and so on) , the staged template 
matching processing means 16a determines a first subregion tR 

15 within the first search region RR, in which first subregion the 

degree of image pattern coincidence with the corresponding first 
template region TR is high . (At thi s time , as an index representing 
the degree of image pattern coincidence, for example, a normalized 
cross correlation value may be employed. ) Furthermore, the staged 

20 template matching processing means 16a calculates a first position 

relationship, which is the corresponding position relationship 
between the first template region TR and the first subregion tR. 
In accordance with the first position relationship, the staged 
template matching processing means 16a calculates a first shift 

25 quantity (+Ax, +Ay) , which is a translation quantity for causing 

the first template region TR to coincide with the first subregion 
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tR . 

(2) Thereafter, as for the second image P2 , in 
accordance with the first shift quantity (+Ax, +Ay) , the staged 
template matching processing means 16a sets a second template 
5 region TR 1 (one of TR • 1 , TR 1 2 , . . . ) , which has its center point 

at the original center point having the coordinates (xO, yO) . 
By way of example, the second template region TR 1 may have a 
rectangular shape having a size of 48 pixels (in the vertical 
direction) * 48 pixels (in the horizontal direction) . Also, as 

10 for the first image PI", the staged template matching processing 

means 16a sets a second search region RR 1 (one of RR 1 1 , RR 1 2 , . . . ) 
having its center point at a position having coordinates (x 1 , 
y ' ) [= (xO+Ax, yO+Ay) ] , which position has been translated by the 
first shift quantity (+Ax, +Ay) from the center point having the 

15 coordinates (xO, yO) in the first subregion tR. The second search 

region RR 1 is larger than the second template region TR 1 and is 
smaller than the first search region RR. By way of example, the 
second search region RR ' may have a rectangular shape having a 
size of 56 pixels (in the vertical direction) x 56 pixels (in the 

20 horizontal direction) . Further, with respect to each of the sets 

of the second template regions TR ■ , TR 1 , . . . and the corresponding 
second search regions RR 1 , RR 1 , ... (a set of TR 1 1 and RR 1 1 , a 
set of TR'2 and RR 1 2 , and so on), the staged template matching 
processing means 16a determines a second subregion tR' within 

25 the second search region RR 1 , in which second subregion the degree 

of image pattern coincidence with the corresponding second 
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template region TR 1 is high. (At this time, as an index 
representing the degree of image pattern coincidence, for example, 
the total sum of the absolute values of the differences between 
pixel values may be employed.) Furthermore, the staged template 
5 matching processing means 16a calculates a second position 

relationship, which is the corresponding position relationship 
between the second template region TR 1 and the second subregion 
tR 1 . In accordance with the second position relationship, the 
staged template matching processing means 16a calculates a second 

10 shift quantity (+Ax f , +Ay ' ) , which is a translation quantity for 

causing the second template region TR 1 to coincide with the second 
subregion tR 1 . 

(3) Thereafter, as for the second image P2 , in 
accordance with the second shift quantity (+Ax' , +Ay 1 ) , the staged 

15 template matching processing means IGasetsa third template region 

TR' 1 (one of TR" 1, TR 11 2 , . . .) , which has its center point at the 
original center point having the coordinates (xO, yO) . By way 
of example, the third template region TR" may have a rectangular 
shape having a size of 16 pixels (in the vertical direction) * 

20 16 pixels (in the horizontal direction) . Also, as for the first 

image PI", the staged template matching processing means 16a sets 
a third search region RR" (one of RR"1, RR"2, ...) having its 
center point at a position having coordinates (x M , y") [=(x'+Ax' , 
y • +Ay 1 ) = (xO+Ax+Ax 1 , yO+Ay+Ay')], which position has been 

25 translatedby the second shift quantity (+Ax' , +Ay ' ) from the center 

point having the coordinates (x 1 , y') in the second subregion 
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tR' . The third search region RR" is larger than the third template 
region TR" and is smaller than the second search region RR ' . By 
way of example, the third search region RR 11 may have a rectangular 
shape having a size of 20 pixels (in the vertical direction) * 
5 20 pixels (in the horizontal direction) . Further, with respect 

to each of the sets of the third template regions TR", TR" , . .. 
and the corresponding third search regions RR", RR" , ... (a set 
of TR"1 and RR"1, a set of TR 11 2 and RR"2, and so on) , the staged 
template matching processing means 16a determines a third 

10 subregion tR" within the third search region RR" , in which third 

subregion the degree of image pattern coincidence with the 
corresponding third template region TR" is high. (At this time, 
as an index representing the degree of image pattern coincidence, 
for example , the total sum of the absolute values of the differences 

15 between pixel values may be employed.) Furthermore, the staged 

template matching processing means 16a calculates a third position 
relationship, which is the corresponding position relationship 
between the third template region TR" and the third subregion 
tR" . In accordance with the third position relationship, the 

20 staged template matching processing means 16a calculates a third 

shift quantity (+Ax", +Ay" ) , which is a translation quantity for 
causing the third template region TR" to coincide with the third 
subregion tR" . 

(4) Thereafter, with respect to the center point of 

25 each ROI (each of R0I1, ROI2, . . .) , the staged template matching 

processing means 16a calculates an ultimate translation quantity 
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(i.e., an ultimate shift quantity) , (+Axi , +Ayi) where i = l , 2, ... 
[= first shift quantity + second shift quantity + third shift 
quantity] . 

Also, in accordance with the translation quantities 
5 (+Axi, +Ayi) of the center points of the respective ROI 1 s in the 

second image P2 , the staged template matching processing means 
16a calculates the translation quantities of all of the pixels 
in each ROI in the second image P2 by performing, for example, 
interpolating operations (linear interpolation, spline 

10 interpolation, Bezier interpolation, or the like) utilizing the 

translation quantities of four neighboring points (the center 
points of the respective ROI's) . 

The partial position matching processing means 16b 
performs warping on only the ROI in the first image PI" and regions 

15 neighboring with the ROI and in accordance with the translation 

quantities of the pixels in the second image P2, which translation 
quantities have been calculated by the staged template matching 
processing means 16a. In this manner, the partial position 
matching processing means 16b performs the matching of positions 

20 of the first image PI" and the second image P2 . Since the template 

region and the search region are set with respect to only the 
ROI 1 s , between which the degree of shift is high, the shift quantity 
(+Axi, +Ayi) between the ultimate corresponding positions of the 
center points of the ROI's, between which the degree of shift 

25 is low, (i.e., the ROI 1 s which need not be sub j ec ted to the posi tion 

re-matching processing) may be uniformly set to be (0, 0). In 
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this manner, the shift quantities of the center points of all 
of the ROI 1 s in the second image P2 may be set. Also, in accordance 
with the thus set shift quantities, the shift quantities of the 
pixels in the ROI ' s and the neighboring regions in the second 
5 image P2 may be calculated with the interpolating operations 

utilizing the shift quantities of the four neighboring points 
(the center points of the ROI's) , which interpolating operations 
will be described later. 

How the image position matching apparatus 10 operates 

10 will be described hereinbelow. 

Firstly, image signals representing the first image 
PI and the second image P2 shown in Figures 3A and 3B, which have 
been acquired in the time series manner, are fed into the global 
position matching processing means 11 of the approximate position 

15 matching processing means 14. As illustrated in Figures 4A and 

4B, with respect to the two received images PI and P2, the global 
position matching processing means 11 takes the currently acquired 
second image P2 as the reference image and performs the affine 
transform on the first image PI. In this manner, the global 

20 position matching processing means 11 matches the position of 

the first image PI with the position of the second image P2 in 
the global manner. 

Thereafter, as illustrated in Figure 5A, the template 
matching processing means 12 sets a plurality of template regions 

25 T2 , T2 , which are rectangular small regions, in the second 

image P2 , which is the current image acting as the reference image. 
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Specifically, of the image PI' (the image having been obtained 
by performing the af f ine transform on the first image PI for the 
global position matching processing) and the second image P2 , 
whose positions have been matched with each other by the global 
5 position matching processing, the second image P2 is taken as 

the reference image. Also, as illustrated in Figure 5B, the 
template matching processing means 12 sets a plurality of search 
regions Rl , Rl , . . - , each of which corresponds to one of the template 
regions T2 , T2 , ... in the second image P2 and is larger than 

10 each of the template regions T, T , in the first image PI*. 

The template matching processing means 12 further determines 
sub regions Tl , Tl , ... in the respective search regions Rl, Rl, . . . , 
in each of which subregions the image pattern approximately 
coincides with the image pattern within the corresponding template 

15 region T in the second image P2 . Furthermore, the template matching 

processing means 12 calculates corresponding position 
relationships (i.e., the shift quantities ) between the subregions 
Tl, Tl, ... in the first image PI 1 and the corresponding template 
regions T2, T2 , ... in the second image P2 . 

20 Thereafter, as illustrated in Figure 6, the local 

position matching processing means 13 performs the nonlinear 
strain transform processing on the entire area of the first image 
PI 1 , which has been obtained from the affine transform, and in 
accordance with the corresponding position relationships between 

25 the subregions Tl, Tl, ... and the corresponding template regions 

T2 , T2 , . . . , which corresponding position relationships have been 
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obtained from the template matching processing means 12 . The 
nonlinear strain transform processing is performed such that the 
subregions Tl , Tl, . . . in the firs t image PI 1 approximately coincide 
with the corresponding template regions T2 , T2 , ... in the second 
5 image P2 . From the nonlinear strain transform processing 

performed on the first image PI* having been obtained from the 
affine transform, the first image PI" is obtained. The position 
of the first image PI" has thus been matched with the position 
of the second image P2 in the local manner. 

10 With the approximate position matching processing 

described above, the positions of the first image P1 M and the 
second image P2 as a whole are matched with each other with a 
certain level of accuracy. However, even after the approximate 
position matching processing has been performed, a shift often 

15 remains between certain local area limited regions. 

Therefore, in the image position matching apparatus 
10, with respect to the first image PI" and the second image P2, 
whose positions have been approximately matched with each other 
by the approximate position matching processing, the region 

20 selecting means 15 performs the processing for selecting the local 

area limited regions, between which the degree of shift is still 
high. Also, the position re -matching proces s ing means 16 performs 
the position re-matching processing on the local area limited 
regions, which have thus been selected by the region selecting 

25 means 15 . 

Specifically, the image signals representing the first 



42 



image PI" and the second image P2, whose positions have been 
approximately matched with each other by the approximate position 
matching processing, are fed into the region selecting means 15. 
As illustrated in Figure 7 and Figures 8A and 8B, the region 
5 selecting means 15 sets the plurality of the ROI 1 s in each of 

the first image PI" and the second image P2 . The ROI 1 s have their 
center points at the corresponding spatial coordinates on the 
first image PI" and the second image P2 and have the identical 
shape. Specifically, ROI1, R0I2, ROI3, . . . are set in the second 

10 image P2 , and ROI1 1 , ROI2 ' , ROI3 1 , ... are set in the first image 

PI". Also, with respect to each of the sets of the corresponding 
ROI ' s (e.g., the set of ROI1 and ROI1 1 , the set of ROI 2 and ROI 2 ■ , 
and so on) in the second image P2 and the first image PI", the 
region selecting means 15 calculates the total sum value Ds of 

15 the absolute values of the pixel value differences between the 

corresponding pixels (i.e., the pixels having the identical 
spatial coordinates) in the corresponding ROI 1 s as the index 
representing the degree of shift between the corresponding ROI ■ s . 
Specifically, as the index representing the degree of shift between 

20 the corresponding ROI 9 s, the region selecting means 15 calculates 

the total sum value Ds [=EE|A(i, j)-B(i, j)|] of the absolute values 
of the differences [=|A(i, j)-B(i, j ) |] between the pixel values 
of the corresponding pixels [the pixel value A(i, j) of the pixel 
within the ROI in the second image P2 and the pixel value B(i, 

25 j ) of the corresponding pixel within the corresponding ROI in 

the first image PI" after being transformed] . 
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Also, the region selecting means 15 compares the thus 
obtained total sum value Ds with the threshold value K having 
been set previously. The region selecting means 15 selects the 
corresponding ROI 1 s , which are associated with the total sum value 
5 Ds larger than the threshold value K (Ds>K) , as the local area 

limited regions, between which the degree of shift is high, and 
which are to be subjected to the position re-matching processing. 
Also, the region selecting means 15 regards the corresponding 
ROI 1 s, which are associated with the total sum value Ds not larger 

10 than the threshold value K (Ds^K) , as the local area limited regions , 

between which the degree of shift is low, and which are not to 
be subjected to the position re-matching processing. More 
specifically, as for the local area limited regions, which are 
not to be subjected to the position re-matching processing, the 

15 ultimate corresponding position relationship (i.e., the shift 

quantity) between the center points of the local area limited 
regions, which are not to be subjected to the position re-matching 
processing, is set to be 0 (zero) in the position re-matching 
processing. In the strict sense, the pixels other than the pixels 

20 at the center points of the local area limited regions, which 

are not to be subjected to the position re-matching processing, 
may be subjected to the calculation of the corresponding position 
relationship (i.e., the shift quantity) and the transform 
processing in the position re-matching processing. 

25 Thereafter, as illustrated in Figure 10, the staged 

template matching processing means 16a performs the processing 
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described below on the received first image Pl n and the second 
image P2 . Specifically, as for the second image P2 acting as the 
reference image, with respect to each ROI in the second image 
P2, the staged template matching processing means 16a sets the 
first template region TR (one of TR1 , TR2 , . ..), which has its 
center point at the center point having the coordinates (xO, yO) 
in the ROI. By way of example, the first template region TR may 
have a rectangular shape having a size of 80 pixels (in the vertical 
direction) x 80 pixels (in the horizontal direction) . Also, as 
for the first image PI", with respect to each ROI in the first 
image PI" , the staged template matching processing means 16a sets 
the first search region RR (one of RR1, RR2 , . . .) , which has its 
center point at the same coordinates (xO, yO) as the coordinates 
of the center point of the first template region TR and is larger 
than the first template region TR. By way of example, the first 
search region RR may have a rectangular shape having a size of 
92 pixels ( in the vertical direction) * 92 pixels ( in the horizontal 
direction) . Further, with respect to each of the sets of the first 
template regions TR, TR, . . . and the corresponding first search 
regions RR, RR, . . . , the staged template matching processing means 
16a determines the first subregion tR within the first search 
region RR, in which first subregion the degree of image pattern 
coincidence with the corresponding first template region TR is 
high. (At this time, as the index representing the degree of image 
pattern coincidence , forexample, the normal ized cross correlation 
value may be employed.) A judgment as to whether the degree of 




image pattern coincidence is high or low may be made by comparison 
with a predetermined threshold value. Furthermore, the staged 
template matching processing means 16a calculates the first 
position relationship, which is the corresponding position 
5 relationship between the first template region TR and the first 

sub region tR . In accordance wi th the first position relationship , 
the staged template matching processing means 16a calculates the 
first shift quantity (+Ax, +Ay) , which is the translation quantity 
for causing the first template region TR to coincide with the 

10 first subregion tR . 

For example, as illustrated in Figures 11A and 11B, 
the image pattern within a first template region TR2 shown in 
Figure 11A, which has its center point at the coordinates (xO, 
yO) , is searched within the corresponding first search region 

15 RR2 shown in Figure 11B, which has its center point at the 

coordinates (xO, yO) . As a result, a first subregion tR2 having 
its center point at the coordinates (x 1 , y 1 ) may be detected as 
the first subregion tR, in which the degree of image pattern 
coincidence with the first template region TR2 is the highest. 

20 In such cases, the f irs t shi f t quanti ty (+Ax, +Ay) may be calculated 

with the formula shown below. 

(+Ax, +Ay)=(x'-xO, y'-yO) 
Thereafter, as for the second image P2 , the staged 
template matching processing means 16a sets the second template 

25 region TR 1 (one of TR 1 1 , TR'2, . . .) , which has its center point 

at the original center point having the coordinates (xO, yO) and 
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is smaller than the first template region TR. By way of example, 
the second template region TR 1 may have a rectangular shape having 
a size of 48 pixels (in the vertical direction) * 48 pixels (in 
the horizontal direction) . Also, as for the first image P1 M , the 
5 staged template matching processing means 16a sets the second 

search region RR 1 (one of RR 1 1 , RR 1 2 , . . . ) having its center point 
at the position having coordinates (x' f y') [= (xO+Ax, yO+Ay) ] , 
which position has been translated by the first shift quantity 
(+Ax, +Ay) from the center point having the coordinates (xO, yO) 

10 in the first subregion tR. The second search region RR 1 is larger 

than the second template region TR 1 and is smaller than the first 
search region RR. By way of example, the second search region 
RR 1 may have a rectangular shape having a size of 56 pixels (in 
the vertical direction) x 56 pixels (in the horizontal direction) . 

15 Further, with respect to each of the sets of the second template 

regions TR' , TR 1 , ... and the corresponding second search regions 
RR 1 , RR ' , ... ( the set of TR ■ 1 and RR'l, the set of TR ' 2 and RR 1 2 , 
and so on) , the staged template matching processing means 16a 
determines the second subregion tR ' within the second search region 

20 RR', in which second subregion the degree of image pattern 

coincidence with the corresponding second template region TR 1 
is high. (At this time, as the index representing the degree of 
image pattern coincidence, for example, the total sum of the 
absolute values of the differences between pixel values may be 

25 employed.) A judgment as to whether the degree of image pattern 

coincidence is high or low may be made by comparison with a 
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predetermined threshold value . Furthermore, the staged template 
matching processing means 16a calculates the second position 
relationship, which is the corresponding position relationship 
between the second template region TR • and the second subregion 
5 tR'. In accordance with the second position relationship, the 

staged template matching processing means 16a calculates the 
second shift quantity ( +Ax 1 , +Ay ' ) , which is the translation 
quantity for causing the second template region TR 1 to coincide 
with the second subregion tR 1 . 

10 For example, as illustrated in Figures 12A and 12B, 

the image pattern within a second template region TR'2 shown in 
Figure 12A, which has its center point at the coordinates (xO, 
yO) , is searched within the corresponding second search region 
RR 1 2 shown in Figure 12B, which has its center point at the 

15 coordinates (x 1 , y'). As a result, a second subregion tR 1 2 having 

its center point at the coordinates (x" , y") may be detected as 
the second subregion tR 1 , in which the degree of image pattern 
coincidence with the second template region TR'2 is the highest. 
In such cases, the second shift quantity (+Ax', +Ay ■ ) may be 

20 calculated with the formula shown below. 

(+Ax', +Ay ' ) = (x M -x 1 , y M -y') 

= ( x " - x 0 - Ax , y 11 - y 0 - Ay ) 
Thereafter, in the same manner as that described above, 
as for the second image P2 , the staged template matching processing 

25 means 16a sets the third template region TR" (one of TR"1, TR 11 2 , . . . ) , 

which has its center point at the original center point having 
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the coordinates (xO, yO) and is smaller than the second template 
region TR ■ . By way of example, the third template region TR 11 may 
have a rectangular shape having a size of 16 pixels (in the vertical 
direction) x 16 pixels (in the horizontal direction) . Also, as 
5 for the first image PI", the staged template matching processing 

means 16a sets the third search region RR 11 (one of RR"1, RR"2, . . . ) 
having its center point at the position having coordinates (x", 
y") [=(x'+Ax', y'+Ay')], which position has been translated by 
the second shift quantity (+Ax', +Ay*) from the center point having 

10 the coordinates (x', y') in the second subregion tR' . The third 

search region RR" is larger than the third template region TR 11 
and is smaller than the second search region RR 1 . By way of example , 
the third search region RR" may have a rectangular shape having 
a size of 20 pixels (in the vertical direction) * 20 pixels (in 

15 the horizontal direction) . Further, with respect to each of the 

sets of the third template regions TR" , TR", ... and the 
corresponding third search regions RR", RR" , . . . (the set of TR 11 1 
and RR"1, the set of TR " 2 and RR 11 2 , and so on) , the staged template 
matching processing means 16a determines the third subregion tR" 

20 within the third search region RR" , in which third subregion the 

degree of image pattern coincidence with the corresponding third 
template region TR" is high. (At this time, as the index 
representing the degree of image pattern coincidence , for example , 
the total sum of the absolute values of the differences between 

25 pixel values may be employed . ) A judgment as to whether the degree 

of image pattern coincidence is high or low may be made by comparison 
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with a predetermined threshold value. Furthermore, the staged 
template matching processing means 16a calculates the third 
position relationship, which is the corresponding position 
relationship between the third template region TR M and the third 
subregion tR" . In accordance with the third position relationship, 
the staged template matching processing means 16a calculates the 
third shift quantity (+Ax" , +Ay M ), which is the translation 
quantity for causing the third template region TR" to coincide 
with the third subregion tR n . 

For example, the image pattern within a third template 
region TR"2, which has its center point at the coordinates (xO, 
yO) , is searched within the corresponding third search region 
RR 11 2 , which has its center point at the coordinates (x" , y 11 ) . 
As a result, a third subregion tR"2 having its center point at 
coordinates (x°, y°) may be detected as the third subregion tR", 
in which the degree of image pattern coincidence with the third 
template region TR"2 is the highest. In such cases, the third 
shift quantity <+Ax" , +Ay") may be calculated with the formula 
shown below. 

(+Ax", +Ay") = (x°-x" , y°-y n ) 
= (x°-xO - Ax- Ax ' , 
y°-yO -Ay-Ay 1 ) 

As described above, with respect to each of the sets 
of the corresponding local area limited regions, the sizes of 
the template region and the search region are reduced in stages, 
and the shift quantity (+Axi, +Ayi), where i=l, 2, between 
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the center points of the corresponding local area limited regions 
in the first image Pl n and the second image P2 is calculated. 
The shift quantity (+Axi, +Ayi), where i = l, 2, . .., may be 
represented by the formula shown below. 
5 (+Axi, +Ayi) = (x°i -xOi , y°i-yOi) 

= (Axi+Ax ' i+Ax"i , 
Ayi+Ay ' i+Ay"i) 

The staged template matching processing means 16a calculates the 
ultimate shift quantity (+Axi, +Ayi) with respect to each of the 

10 center points of the local area limited regions in the second 

image P2 . As described above, with respect to each of the center 
points of the local area limited regions, which were not selected 
as the local area limited regions to be subjected to the position 
re-matching processing, the shift quantity (+Axi, +Ayi) is 

15 uniformly set to be (0, 0) . 

Also, in accordance with the shift quantities (+Axi, 
+Ayi) of the center points of the respective ROI's in the second 
image P2, the staged template matching processing means 16a 
calculates the shift quantities of all of the pixels in each ROI 

20 in the second image P2 by performing for example, linear 

interpolating operations utilizing the shift quantities of the 
four neighboring points (the center points of the respective ROI 1 s) 
Alternatively, in lieu of the linear interpolating operations, 
one of other interpolating operations, such as spline 

25 interpolation, Bezier interpolation, and NURBS interpolation 

(weighted B spline interpolation, may be employed. Also, the 
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number of the neighboring points utilized in the interpolating 
operations is not limited to four and may be set appropriately 
in accordance with the kind of the interpolating operations, and 
the like. (For example, 16 neighboring points may be utilized. ) 
5 Specifically, the template regions TR1 , TR2 , . . . , TR11, 

TR12, ... in the second image P2 may be arrayed in the pattern 
illustrated in Figure 13. In such cases, as described above, the 
shift quantity of the center point of the template region TR1 
is (+Axl, +Ayl) [i.e., (+Axi, +Ayi) , where i=l] , and the shift 

10 quantity of the center point of the template region TR2 is (+Ax2, 

+Ay2 ) . Also , the shift quantity of the center point of the template 
region TR11 is (+Axll, +Ayll) , and the shift quantity of the center 
point of the template region TR12 is (+Axl2, +Ayl2) . As illustrated 
in Figure 14, the shift quantity (+Ax(x), +Ay(y)) of each pixel 

15 having the coordinates (x, y) , which pixel falls within the range 

surrounded by the four center points of the four template regions 
TR1, TR2 , TR11, and TR12 , may be calculated with the linear 
interpolating operations in the two-dimensional plane. The 
calculations may be made with the formulas shown below. 

20 Ax(x)=(l-u) (1-v) Axl+u (1-v) Ax2 

+ (1 -u) vAxll+uvAxl2 
Ay(y)=(l-u) (1-v) Ayl+u (1-v) Ay2 
+ ( 1 -u) vAyll+uvAyl2 
In the manner described above, the shift quantities 

25 (+Ax (x) , +Ay (y) ) of the pixels having the coordinates (x, y) , which 

pixels fall within the selected ROI in the second image P2 and 
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the regions neighboring with the selected ROI , are calculated 
by the staged template matching processing means 16a. In 
accordance with the thus calculated shift quantities (+Ax(x), 
+ Ay(y)), the partial position matching processing means 16b of 
5 the position re-matching processing means 16 performs the warping 

on the pixels, which fall within the selected ROI in the first 
image PI" and the regions neighboring with the selected ROI. In 
this manner, the partial position matching processing means 16b 
performs the matching of positions of the first image PI" and 

10 the second image P2 . 

An image Pl° (hereinbelow referred to as the first image 
Pl°) is obtained from the warping performed on a certain area of 
the first image PI". The pixel array in the first image Pl° is 
different from the pixel array in the second image P2 . Therefore, 

15 in cases where it is considered that, for example, a subtraction 

process will be performed on corresponding pixels in the two images , 
and the image will be displayed, it is necessary for the pixel 
array in the first image Pl° to be caused to coincide with the 
pixel array in the second image P2 , which acts as the reference 

20 image. Accordingly, as illustrated in Figure 15, the partial 

position matching processing means 16b performs linear 
interpolating processing in the two-dimensional plane on the 
pixels in the first image Pl° and calculates the pixel values with 
the formula shown below. 

25 g(x+Ax, y+Ay) = (1-u) (1-v) g (xl, yl) 

+u(l-v)g(x2, y2) 
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+ (1-u) vg(xll, yll) 

+uvg(xl2, yl2) 
A second embodiment of the apparatus for matching 
positions of images in accordance with the present invention will 
be described hereinbelow. Basically, the second embodiment of 
the apparatus for matching positions of images in accordance with 
the present invention is constituted and operates in the same 
manner as that in the first embodiment described above, except 
for the features described below. The second embodiment of the 
apparatus for matching positions of images in accordance with 
the present invention has the general constitution as illustrated 
in Figure 2. Figure 17 shows a processing flow performed by the 
staged template matching processing means 16a of the position 
re-matching processing means 16 in the second embodiment. 

In the second embodiment, the staged template matching 
processing means 16a of the position re-matching processing means 
16 operates in the manner described below. Specifically, in the 
same manner as that in the staged template matching processing 
means 16a employed in the first embodiment described above, in 
each of the stages associated with the operation for reducing 
the sizes of the template region and the search region in stages, 
the staged template matching processing means 16a in the second 
embodiment calculates the corresponding position relationship 
of each stage. Thereafter, the staged template matching 
processing means 16a in the second embodiment performs spring 
constraint processing and calculates a spring constraint processed 



corresponding position relationship in each stage. The spring 
constraint processed corresponding position relationship, which 
has thus been calculated, is taken as the corresponding position 
relationship in each stage. Also, the staged template matching 
5 processing means 16a iterates the operations for successively 

calculating the spring constraint processed corresponding 
position relationships in the respective stages and thereby 
calculates the ultimate corresponding position relationship. 

More specifically, as illustrated in Figure 17, the 

10 staged template matching processing means 16a in the second 

embodiment performs the processing described below. 

(1) As for the second image P2 acting as the reference 
image, with respect to each ROI in the second image P2 , the staged 
template matching processing means 16a sets the first template 

15 region TR (one of TR1, TR2 , . . .) , which has its center point at 

the center point having coordinates (x0 # yO) in the ROI. By way 
of example, the first template region TR may have a rectangular 
shape having a size of 80 pixels (in the vertical direction) x 
80 pixels (in the horizontal direction) . Also, as for the first 

20 image PI", with respect to each ROI in the first image PI", the 

staged template matching processing means 16 a sets the first search 
region RR (one of RR1 , RR2 , ...), which has its center point at 
the same coordinates (xO, yO) as the coordinates of the center 
point of the first template region TR and is larger than the first 

25 template region TR . By way of example, the first search region 

RR may have a rectangular shape having a size of 92 pixels (in 
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the vertical direction) x 92 pixels (in the horizontal direction) . 
Further, with respect to each of the sets of the first template 
regions TR, TR, . . . and the corresponding first search regions 
RR, RR, . . . (the set of TR1 and RR1 , the set of TR2 and RR2 , and 
so on) , the staged template matching processing means 16a 
determines the first subregion tR within the first search region 
RR, in which first subregion the degree of image pattern coincidence 
with the corresponding first template region TR is high. (At this 
time, as the index representing the degree of image pattern 
coincidence, for example, the normalized cross correlation value 
may be employed.) Furthermore, the staged template matching 
processing means 16a calculates the first position relationship, 
which is the corresponding position relationship between the first 
template region TR and the first subregion tR . In accordance with 
the first position relationship, the staged template matching 
processing means 16a calculates the first shift quantity (+Ax, 
+Ay) , which is the translation quantity for causing the first 
template region TR to coincide with the f irst subregion tR . Also, 
the staged template matching processing means 16a calculates a 
shift quantity with respect to each of sets of small regions, 
which have been set around the first template region TR, and the 
corresponding small regions, which have been set around the first 
subregion tR . Further, the staged template matching processing 
means 16a performs the spring constraint processing in accordance 
with the first shift quantity (+Ax, +Ay) and the shift quantities 
having been calculated with respect to the sets of the small regions . 



In this manner, the staged template matching processing means 
16a calculates a first spring constraint processed shift quantity 
(+AX, +AY) . 

(2) Thereafter, as for the second image P2 , in 
5 accordance with the first spring constraint processed shift 

quantity (+AX, +AY) , the staged template matching processing means 
16a sets a second template region TR 1 (one of TR ' 1 , TR ■ 2 , ...), 
which has its center point at the original center point having 
the coordinates (xO, yO) . By way of example, the second template 

10 region TR 1 may have a rectangular shape having a size of 48 pixels 

(in the vertical direction) * 48 pixels (in the horizontal 
direction) . Also, as for the first image PI", the staged template 
matching processing means 16a sets a second search region RR 1 
(one of RR ■ 1 , RR 1 2 , . . . ) having its center point at a position 

15 having coordinates (x 1 , y') [= (xO+AX, yO+AY) ] , which position has 

been translated by the first spring constraint processed shift 
quantity (+AX, +AY) from the center point having the coordinates 
(xO, yO) in the first subregion tR. The second search region RR 1 
is larger than the second template region TR 1 and is smaller than 

20 the first search region RR. By way of example, the second search 

region RR 1 may have a rectangular shape having a size of 56 pixels 
(in the vertical direction) x 56 pixels (in the horizontal 
direction) . Further, with respect to each of the sets of the second 
template regions TR 1 , TR 1 , . . . and the corresponding second search 

25 regions RR 1 , RR ' , ... (a set of TR 1 1 and RR'l, a set of TR' 2 and 

RR'2, and so on), the staged template matching processing means 
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16a determines a second subregion tR 1 within the second search 
region RR' , in which second subregion the degree of image pattern 
coincidence with the corresponding second template region TR ' 
is high. (At this time, as an index representing the degree of 
image pattern coincidence, for example, the total sum of the 
absolute values of the differences between pixel values may be 
employed. ) Furthermore, the staged template matching processing 
means 16a calculates a second position relationship, which is 
the corresponding position relationship between the second 
template region TR 1 and the second subregion tR 1 . In accordance 
with the second position relationship, the staged template 
matching processing means 16a calculates a second shift quantity 
( +Ax 1 , +Ay 1 ) , which is a translation quantity for causing the second 
template region TR ' to coincide with the second subregion tR ' . 
Also, the staged template matching processing means 16a calculates 
a shift quantity with respect to each of sets of small regions, 
which have been set around the second template region TR 1 , and 
the corresponding small regions, which have been set around the 
second subregion tR'. Further, the staged template matching 
processing means 16a performs the spring constraint processing 
in accordance with the second shift quantity (+Ax' , +Ay' ) and the 
shift quantities having been calculated with respect to the sets 
of the small regions . In this manner, the staged template matching 
processing means 16a calculates a second spring constraint 
processed shift quantity (+AX 1 , +AY ' ) . 

(3) Thereafter, as for the second image P2 , in 
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accordance with the second spring constraint processed shift 
quantity ( +AX 1 , +AY 1 ) , the staged template matching processing 
means 16a sets a third template region TR" (one of TR 11 1 , TR 11 2 , . . . ) , 
which has its center point at the original center point having 
5 the coordinates (xO, yO) . By way of example, the third template 

region TR" may have a rectangular shape having a size of 16 pixels 
(in the vertical direction) x 16 pixels (in the horizontal 
direction) . Also, as for the first image PI", the staged template 
matching processing means 16a sets a third search region RR" (one 

10 of RR"1, RR"2, . . .) having its center point at a position having 

coordinates (x" , y") [-(x'+AX 1 , y ' +AY 1 ) = (xO+AX+AX 1 , yO+AY+AY ' ) ] , 
which position has been translated by the second spring constraint 
processed shift quantity (+AX*, +AY ' ) from the center point having 
the coordinates (x 1 , y') in the second subregion tR 1 . The third 

15 search region RR" is larger than the third template region TR 11 

and is smaller than the second search region RR 1 . Byway of example , 
the third search region RR 11 may have a rectangular shape having 
a size of 20 pixels (in the vertical direction) x 20 pixels (in 
the horizontal direction) . Further, with respect to each of the 

20 sets of the third template regions TR 11 , TR " , ... and the 

corresponding third search regions RR" , RR" , ... (a set of TR"1 
and RR"1, a set of TR"2 and RR"2, and so on) , the staged template 
matching processing means 16a determines a third subregion tR" 
within the third search region RR", in which third subregion the 

25 degree of image pattern coincidence with the corresponding third 

template region TR" is high. (At this time, as an index 
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representing the degree of image pattern coincidence , for example, 
the total sum of the absolute values of the differences between 
pixel values may be employed.) Furthermore, the staged template 
matching processing means 16a calculates a third position 
5 relationship, which is the corresponding position relationship 

between the third template region TR" and the third subregion 
tR". In accordance with the third position relationship, the 
staged template matching processing means 16a calculates a third 
shift quantity (+Ax", +Ay M ), which is a translation quantity for 

10 causing the third template region TR" to coincide with the third 

subregion tR" . Also, the staged template matching processing 
means 16a calculates a shift quantity with respect to each of 
sets of small regions , which have been set around the third template 
region TR", and the corresponding small regions, which have been 

15 set around the third subregion tR" . Further, the staged template 

matching processing means 16a performs the spring constraint 
processing in accordance with the third shift quantity (+Ax", +Ay") 
and the shift quantities having been calculated with respect to 
the sets of the small regions . In this manner, the staged template 

20 matching processing means 16a calculates a third spring constraint 

processed shift quantity (+AX", +AY"). 

(4) Thereafter, with respect to the center point of 
each ROI (each of ROI1 , ROI2 , . . . ) , the staged template matching 
processing means 16a calculates an ultimate translation quantity 

25 ( i . e . , an ultimate shift quantity) , ( +AXi , +AYi ) where i = l , 2 , ... 

[ = first spring constraint processed shift quant i ty + second spring 
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constraint processed shift quantity + third spring constraint 
processed shift quantity] . 

How the spring constraint processing is performed will 
be described hereinbelow. 
5 As illustrated in Figure 18, with respect to a template 

ROI (in this case, TR12) , which has its center point at the center 
point having the coordinates (xO, yO) in the local area limited 
region associated with the high degree of shift, and a group of 
surrounding template ROI's (in this case, eight regions of TR1, 

10 TR2, TR3, TR11, TR13, TR23, TR22, and TR21), which surround the 

template ROI (TR12) in a onefold manner, a shift vector (i.e., 
a shift quantity) of each of the surrounding template ROI's and 
the shift vector (i.e., the shift quantity) of the template ROI 
(TR12) , which is a center region, are constrained with each other 

15 by a virtual spring. In this manner, the shift vectors of the 

surrounding template ROI's are balanced with one another by 
constraining forces of the respective springs. 

Specifically, the shift vector of the template ROI 
(TR12), which is the center region, may be represented by (x(2, 

20 2) , y (2 , 2) ) . The shift vector of each of the surrounding template 

ROI 1 s may be represented by (x(2+k, 2 + 1), y (2+k, 2 + 1)). Also, 
a spring constant between the shift vector of the template ROI 
(TR12) , which is the center region, and the shift vector of each 
of the surrounding template ROI's may be represented by a(k, 1) . 

25 In such cases, by the balance of the spring constraining forces, 

the shift quantity of the template ROI (TR12) , which is the center 
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region, is corrected by Ax(2, 2) in the x direction and by Ay(2, 
2) in the y direction. More specifically, a pulling phenomenon 
in accordance with the differences in shift vector occurs due 
to the spring constraining forces between the shift vector (x(2, 
2), y(2, 2)) of the template ROI (TR12 ) , which is the center region, 
and the shift vectors (x(2+k, 2 + 1), y(2+k, 2 + 1)) of the eight 
surrounding template ROI 1 s . As a result, the shift vector (x(2, 
2), y(2, 2)) of the template ROI (TR12 ) , which is the center region, 
is corrected by a vector (Ax (2, 2), Ay (2, 2)). The correction 
quantity (Ax(2, 2), Ay(2, 2)) maybe represented by Formulas (1) 
and (2) shown below. 

Ax(2,2)= v [a(k,/)x{%(2 + k,2+i)-x(2 J 2)} ] (1) 
Ay(2,2)=2 [a(M)x{y(2 + k,2 + i?)-y( 2 3 2 )} ] (2) 

Also, a shift vector (nx(2, 2), ny(2, 2)) after being 
corrected by the correction quantity may be calculated with 
Formulas (3) and (4) shown below. 

nx(2,2) = x(2 J 2) + b-Ax(2,2) (3) 
ny(2,2)-y(2,2)+b-Ay(2 J 2) (4) 

where b represents a fixed number (0<b^l). 

As described above, in cases where the template ROI 

(TR12) is taken as the center region, the shift vector of the 
template ROI (TR12) may be corrected by the correction quantity 

(Ax (2 , 2 ) , Ay ( 2 , 2 ) ) . As for a shift vector (x(i, j), y(i, j)) 
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of a template ROI (TRij) , a correction quantity (Ax(i, j) , Ay { i , 
j)) may be represented by Formulas (5) and (6) shown below. 

g 

Ax(i,j)= s [a(k,/)x{x(i + k J j+i)-x(i J j)} ] (5) 

g 

5 Ay(i j)- 2 [a(k^)x{y(i + k,j+i)-y(i f j)} ] (6) 

M 

Also, a shift vector (nx ( i , j ) , ny ( i , j ) ) of each template 
ROI (TRij) after being corrected by the correction quantity (Ax(i, 
j), Ay(i, j)) may be calculated with Formulas (7) and (8) shown 
10 below. 

nx(i j)-x(i,j)+b-Ax(i,j) (7) 
ny(i j)-y(ij)+b-Ay(i j) (8) 

A total sum £a(k, 1) of the spring constants a(k, 1) 
15 of the eight springs with respect to one center region should 

preferably be normalized to 1. 

In the embodiment described above, as the surrounding 
template ROI's, which constrain one template ROI located at the 
center region, the eight template ROI's surrounding the center 
20 region in a onefold manner are employed. Alternatively, one 

template ROI located at the center region may be constrained with 
24 template ROI's, which contain the eight template ROI's 
surrounding the center region in a onefold manner and 16 template 
ROI's located on the side outward from the eight template ROI's. 
25 In general, in cases where the number of the constraining template 

ROI's is represented by Nc , Formulas (5) and (6) shown above may 
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be expressed as Formulas (9) and (10) shown below. 

Nc 

Ax(i,j)= 2 [a(k,/)x{x(i + k,j + i)-x(i j)} ] (9) 
M 

Nc 

Mij)-s[a(^My(i+M+^)-y(i j)} ] do) 

5 M 

As described above, the shift vector of each template 
ROI and the shift vectors of the surrounding template ROI 1 s are 
pulled by each other with the spring constraining forces and are 
thereby balanced. In this manner, the shift vector is capable 
10 of being smoothed . Therefore, even if problems occur accidentally 

in that the shift vector of the template ROI is directed to a 
direction markedly different from the directions of the shift 
vectors of the surrounding template ROI's or has a shift quantity 
markedly larger than the shift quantities of the surrounding 
15 template ROI's, the adverse effects of the problems are capable 

of being suppressed. 

In the second embodiment, the staged template matching 
processing means 16a calculates the ultimate shift quantity ( +AXi , 
+AYi) of the center point of each local area limited region in 
20 the second image P2 by utilizing the spring constraint processing 

in the manner described above. The features in the second 
embodiment other than the processing in the staged template 
matching processing means 16a are the same as those in the first 
embodiment described above. 
25 In the second embodiment described above, the spring 

constraint processing is employed in all of the stages in the 
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staged template matching processing means 16a of the position 
re-matching processing means 16. However, the image position 
matching apparatus in accordance with the present invention is 
not limited to the second embodiment. For example, the spring 
5 constraint processing may be employed in one stage or two stages 

among the stages in the staged template matching processing means 
16a of the position re-matching processing means 16. 

As described above, with each of the two embodiments 
of the image position matching apparatus in accordance with the 

10 present invention, the approximate position matching processing 

is performed with respect to the entire areas of the two images 
PI and P2 , and only the certain local area limited regions , between 
which the degree of shift ishigh even after the approximate position 
matching processing has been performed, are selected. Also, the 

15 position re-matching processing is performed with respect to only 

the selected local area limited regions and the regions neighboring 
with the selected local area limited regions. In this manner, 
in cases where a shift remains between certain local area limited 
regions after the approximate position matching processing has 

20 been performed, the degree of shift between the local area limited 

regions, between which the degree of shift was high, is capable 
of being suppressed. Therefore, the degree of shift between the 
entire areas of the images is capable of being suppressed more 
reliably than with the conventional image position matching 

25 techniques, and the position matching accuracy is capable of being 

enhanced. Accordingly, the two images Pl° and P2 , whose posi tions 
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have been accurately matched with each other by the image position 
matching apparatus in accordance with the present invention, may 
be utilized for inter-image operation processing, in which the 
operation processing is performed on corresponding pixels in the 
5 two images Pl° and P2 . In this manner, an inter- image operation 

processed image may be obtained . By way of example , as illustrated 
in Figure 16C, a subtraction image Psu may be obtainedby subtracting 
the image Pl° shown in Figure 16A from the second image P2 shown 
in Figure 16B. In Figures 16B and 16C, a reference character L 

10 represents a tumor pattern appearing in the second image P2 and 

the subtraction image Psu. Intheinter - image operation processed 
image, which has thus been obtained, an artifact occurring due 
to a shift between the two images has been suppressed markedly. 
In this manner, an image appropriate for making a diagnosis of 

15 an illness, or the like, is capable of being obtained. 
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